Dynamic Quantization using Spike Generation Mechanisms

This paper introduces a neuro-inspired co-ding/decoding mechanism of a constant real value by using a Spike Generation Mechanism (SGM) and a combination of two Spike Interpretation Mechanisms (SIM). One of the most efficient and widely used SGMs to encode a real value is the Leaky-Integrate and Fire (LIF) model which produces a spike train. The duration of the spike train is bounded by a given time constraint. Seeking for a simple solution of how to interpret the spike train and to reconstruct the input value, we combine two different kinds of SIMs, the time-SIM and the rate-SIM. The time-SIM allows a high quality interpretation of the neural code and the rate-SIM allows a simple decoding mechanism by couting the spikes. The resulting coding/decoding process, called the Dual-SIM Quantizer (Dual-SIMQ), is a non-uniform quantizer. It is shown that it coincides with a uniform scalar quantizer under certain assumptions. Finally, it is also shown that the time constraint can be used to control automatically the reconstruction accuracy of this time-dependent quantizer

Retina-inspired image and video coding

Cette thèse vise à proposer une nouvelle architecture de codage vidéo qui s’inspire du système visuel des mammifères et de la rétine. La rétine peut être considérée comme une machine intelligente qui traite le stimulus visuel de façon très efficace. De ce fait, elle représente une grande source d'inspiration pour développer de nouveaux systèmes de traitement d’image. Il y a plusieurs raisons pour cela : elle consomme peu d’énergie, elle traite des entrées haute résolution et sa façon de transformer et d’encoder de manière dynamique le stimulus visuel dépasse les normes actuelles. Nous avons souhaité étudier et proposer un codec vidéo inspiré de la rétine. L’algorithme proposé a été appliqué à un flux vidéo d'une manière simple, suivant le principe des standards de codage MJPEG ou MJPEG2000. Cette approche permet au lecteur d’étudier et d’explorer tous les avantages du traitement dynamique de la rétine en termes de compression et de traitement d’image. La performance actuelle du codec que nous avons développé est très prometteuse. Les résultats montrent des performances supérieures à MJPEG pour des débits inférieurs à 100 kbps et MPEG-2 pour des débits supérieurs à 70 kpbs. De plus, à faibles débits, le codec proposé décrit mieux le contenu de la scène d’entrée. De nombreuses perspectives sont proposées afin d'améliorer ce codec inspiré de la rétine qui semblent conduire à un nouveau paradigme en compression vidéo.The goal of this thesis is to propose a novel video coding architecture which is inspired by the mammalian visual system and the retina. If one sees the retina as a machine which processes the visual stimulus, it seems an intelligent and very efficient model to mimic. There are several reasons to claim that, first of all because it consumes low power, it also deals with high resolution inputs and the dynamic way it transforms and encodes the visual stimulus is beyond the current standards. We were motivated to study and release a retina-inspired video codec. The proposed algorithm was applied to a video stream in a very simple way according to the coding standards like MJPEG or MJPEG2000. However, this way allows the reader to study and explore all the advantages of the retina dynamic processing way in terms of compression and image processing. The current performance of the retina-inspired codec is very promising according to some final results which outperform MJPEG for bitrates lo wer than 100 kbps and MPEG-2 for bitrates higher than 70 kpbs. In addition, for lower bitrates, the retina-inspired codec outlines better the content of the input scene. There are many perspectives which concern the improvement of the retina-inspired video codec which seem to lead to a groundbreaking compression architecture. Hopefully, this manuscript will be a useful tool for all the researchers who would like to study further than the perceptual capability of the mammalian visual system and understand how the structure and the functions of the retina can in practice improve the coding algorithms

Compression d'images et de vidéos inspirée du fonctionnement de la rétine

The goal of this thesis is to propose a novel video coding architecture which is inspired by the mammalian visual system and the retina. If one sees the retina as a machine which processes the visual stimulus, it seems an intelligent and very efficient model to mimic. There are several reasons to claim that, first of all because it consumes low power, it also deals with high resolution inputs and the dynamic way it transforms and encodes the visual stimulus is beyond the current standards. We were motivated to study and release a retina-inspired video codec. The proposed algorithm was applied to a video stream in a very simple way according to the coding standards like MJPEG or MJPEG2000. However, this way allows the reader to study and explore all the advantages of the retina dynamic processing way in terms of compression and image processing. The current performance of the retina-inspired codec is very promising according to some final results which outperform MJPEG for bitrates lo wer than 100 kbps and MPEG-2 for bitrates higher than 70 kpbs. In addition, for lower bitrates, the retina-inspired codec outlines better the content of the input scene. There are many perspectives which concern the improvement of the retina-inspired video codec which seem to lead to a groundbreaking compression architecture. Hopefully, this manuscript will be a useful tool for all the researchers who would like to study further than the perceptual capability of the mammalian visual system and understand how the structure and the functions of the retina can in practice improve the coding algorithms.Cette thèse vise à proposer une nouvelle architecture de codage vidéo qui s’inspire du système visuel des mammifères et de la rétine. La rétine peut être considérée comme une machine intelligente qui traite le stimulus visuel de façon très efficace. De ce fait, elle représente une grande source d'inspiration pour développer de nouveaux systèmes de traitement d’image. Il y a plusieurs raisons pour cela : elle consomme peu d’énergie, elle traite des entrées haute résolution et sa façon de transformer et d’encoder de manière dynamique le stimulus visuel dépasse les normes actuelles. Nous avons souhaité étudier et proposer un codec vidéo inspiré de la rétine. L’algorithme proposé a été appliqué à un flux vidéo d'une manière simple, suivant le principe des standards de codage MJPEG ou MJPEG2000. Cette approche permet au lecteur d’étudier et d’explorer tous les avantages du traitement dynamique de la rétine en termes de compression et de traitement d’image. La performance actuelle du codec que nous avons développé est très prometteuse. Les résultats montrent des performances supérieures à MJPEG pour des débits inférieurs à 100 kbps et MPEG-2 pour des débits supérieurs à 70 kpbs. De plus, à faibles débits, le codec proposé décrit mieux le contenu de la scène d’entrée. De nombreuses perspectives sont proposées afin d'améliorer ce codec inspiré de la rétine qui semblent conduire à un nouveau paradigme en compression vidéo

Efficiency of the Bio-Inspired Leaky Integrate-and-Fire Neuron for Signal Coding

International audienceThe goal of this paper is to investigate whether purely neuro-mimetic architectures are more efficient for signal compression than architectures that combine neuroscience and state-of-the-art models. We are motivated to produce spikes, using the LIF model, in order to compress images. Seeking for solutions to improve the efficiency of the LIF in terms of the memory cost, we compare two different quantization approaches; the Neuro- inspired Quantization (NQ) and the Conventional Quantization (CQ). We present that the purely neuro-mimetic architecture, that combines the LIF and the NQ, is more efficient in terms of the rate-distortion trade-off due to the dynamic properties embedded in these neuro-mimetic models. To achieve this goal, we first study which are the dynamic properties of the recently released (NQ) which is an intuitive way of counting the number of spikes. We show that the observation window and the resistance are the most important parameters of NQ that strongly influence its behavior that ranges from non-uniform to uniform. As a result, the NQ is more flexible than the CQ when it is applied to real data while for the same bit rate it ensures higher reconstruction quality

Retinal-inspired filtering for dynamic image coding

International audienc

Bio-inspired quantization

National audienceThis paper introduces a novel retina-inspired sparse representation which is applied to temporally constant 2D inputs. This architecture consists of the recently released retina-inspired filtering which mimics the transformation of the visual stimulus into current as it takes place in the retina. This transform is very redundant. As a result, we propose the Perfect Leaky Integrate and Fire (Perfect-LIF) as a model which sparsifies the over-complete retina-inspired decomposition mimicking the spike generation mechanisms of the neurons. The Perfect-LIF is a thresholding function based on a time-depended deadzone. Numerical results show the efficiency of our architecture which provides almost equivalent reconstruction results between the over-complete and the sparse representation of the input image.Cet article présente une nouvelle représentation parcimonieuse des images inspirée de la rétine. Cette représentation consiste en un filtrage inspiré par le fonctionnement de la rétine qui imite la transformation du stimulus visuel en un signal spatio-temporel fortement redondant. Nous proposons d’utiliser un codage du type “Intègre-et-Tire” parfait pour réduire la redondance de cette transformation en mimant la génération des potentiels d’action neuronaux. L’encodage “Intègre-et-Tire” parfait est équivalent à un seuillage basé sur une zone morte dépendante du temps d’observation de l’image. Les résultats numériques montrent l’efficacitè de cette représentation parcimonieuse qui fournit des résultats de reconstruction presque équivalents à ceux de la représentation redondante

Neuro-Inspired Quantization

International audienc

Signal reconstruction from a bio-inspired event-based code

National audienc

Retina inspired Video Codec

International audienceIn this paper, we aim to propose a video codec based on the novel retina-inspired filter and retina-inspired quantizer which both perform according to the early visual system. The recently released non-separable spatiotemporal OPL retina-inspired filter enables to progressively extract different kind of information from the input signal which is the sequence of pictures of a video stream. This retina inspired transform has been proven to be a redundant frame which ensures a perfect reconstruction when no quantization appears. The reduction of this redundancy is achieved by a quantization which is inspired by the spike generation mechanism of ganglion cells. This mechanism has been approximated by the Rank Order Coder (ROC) and the Leaky-Integrate and Fire (LIF) models. The ROC model encodes the rank of the spikes and it has been proposed as a complete and very efficient codec for still-images. However, its limitations concerning the reconstruction method forced us to focus our attention on LIF which encodes the spike delays. We approximate the LIF by a scalar quantizer with a dead-zone. This is the first attempt to build a complete retina-inspired video codec which gives promising reconstruction results at low bitrate and high reconstruction quality

Video analysis and synthesis based on a retinal-inspired frame

International audienc